Information coding in image signals has a broad array of applications, and there many techniques for accomplishing it. The information coding may be visible or hidden from human perception when the imagery conveying the data is rendered. It may be incorporated in various forms of electromagnetic-signals that convey visual content, such as digital images or video. It may also be transferred to a physical object through image projection, printed inks or other data writing techniques, such as engraving, etching, molds, etc.
Images with coded information are captured from objects or signals and processed to extract the coded information. The capture process may use special purpose devices, such as bar coding reading equipment, or general purpose image capture devices, such as cameras and scanners. The captured image signal is then processed to recover the coded information. The coded information, itself may encompass various forms of information, including other images (e.g., images of text or graphics encoded within a host image), audio, or machine readable data streams.
Digital watermarking is an information coding field that applies to many forms of signals, with one significant subset being images. In this field, sometimes more generally referred to as steganography, researchers have long sought to increase the capacity and robustness of information capacity of the channel within an image signal, while making the presence of information imperceptible to humans, and the signal processing to encode and decode this information computationally efficient. While the field is well-developed, and various implementations have been widely deployed, these fundamental challenges remain.
The desire to achieve imperceptibility has led researchers to study the human visual system to determine ways in which information may be encoded efficiently, reliably, yet imperceptibly. Practical applications dictate constraints that make this task difficult. One can achieve imperceptibility, even with well-known message coding symbologies like bar codes, by printing or projecting signals in a physical media that are outside human perception, yet can be detected by special purpose illumination and image capture, e.g., using Ultraviolet or Infrared related techniques. These techniques, of course, tend to preclude broad application to devices that operate in the visible spectrum. Preferably, to have broader application, the information coding technique should not require special materials or illumination-capture techniques.
The study of the human visual system, particularly in field of digital watermarking, has led to the development of various masking models. These models are used to determine the ability of image signals to mask other image signals. They are adapted for various types of applications, such as electronic displays, print quality, and are also adapted for static and time varying images (video). Various attributes of image content are evaluated to assess masking capability of an image for certain other image signal types. Some attributes include signal activity or noisiness, color, resolution, contrast, and motion. These attributes, have various masking capability on image content with like or different quantities of these attributes. Various signal transforms are used to convert an image into transform domain coefficients, such as various forms of decomposition into coefficients or sub-bands, to examine various image attributes at spatial frequencies.
One goal of this particular research, particularly as it relates to encoding auxiliary information, is to determine an information channel within an image in which changes to the image may be made with little or no impact on the visual appearance of the image to a human. Preferably, this channel should remain within the space of image variables that are typically available and controllable to enable auxiliary information encoding using digital signal processing on standard image formats. For example, color image signals are typically represented as arrays of pixels, with each pixel represented using three or more image variables, such as Red, Green, Blue values (RGB), or Cyan, Magenta, Yellow, and black (CMYK). Some formats define pixels in terms of luminance, chrominance, hue, intensity and/or saturation. There are a variety of formats for digital representation of image signals based on these or other variables. Information coding that operates on images of these formats derives masking models and adjusts values of these variables or parameters derived from them, to encode auxiliary information.
For certain application domains, such as information channels in printed images, additional degrees of freedom may be obtained by controlling the selection and application of inks. Printer manufacturers can, for example, build the capability to control the depth, shape, spatial arrangement of ink over a unit of area corresponding to a digital pixel or groups of pixels. The degrees of freedom expand across the colors, and combination of inks that represent each of them. Various half-toning techniques are used to convert digital pixel values into a format for controlling application of ink to various types of paper. See, for example, our application, 61/719,920, AUXILIARY DATA EMBEDDING IN RASTER IMAGE PROCESSOR AND IMAGE RECOGNITION, filed Oct. 29, 2012, and its counterpart, Ser. No. 13/789,126, published as US Patent Application Publication No. 2014-0119593, which are hereby incorporated by reference.
Likewise, greater control may also be provided over conversion of image signals into analog form via display or projection. As in the case of printing, this affords greater flexibility in the encoding technology. However, access to this control is not always available. Information coding that does not rely on this additional control over how images are represented and rendered have the advantage of broader applicability across a range of devices and systems.
Another constraint for many applications, as noted above, is that the information coding technique must produce an information signal in the image that can be “seen” by the capture device. Use of special inks have value for some applications, like counterfeit deterrence, yet do not allow the signal to be communicated through typical cameras that operate in the visible light range (e.g., those now ubiquitous on mobile phones, tablets and PCs).
The application of color science has offered promising advances in information coding in images. Changes in some colors are more noticeable than others to the typical human. By combining these phenomena with other masking techniques, additional information signals may be encoded within a desired perceptibility threshold. This observation has led to the development of image coding techniques that exploit it. See, for example, our US Patent Publication 20110216936.
Within the field of color science, metamerism refers to the matching of colors that appear similar to humans, yet have different spectral power distributions. This property may have promise in information coding within images, as it offers the potential of using colors with different spectral distributions as means to encode information within an image. These colors appear the same to humans, yet have differences that are detectable given the appropriate capture and signal processing that can discern the signal in the differences between spectral distributions of colors sensed from an object or image. In order to make this available using a wide array of image types, there is need for techniques to encode auxiliary information in spectral differences that can apply to standard image formats and commonly used image rendering technology (e.g., image printers, displays, projection systems, etc.). Further, there is a need for sensors and/or signal processing techniques that can discern spectral differences sufficiently to recover the information encoded in these differences.
In this document, we describe a variety of inventive methods and related hardware components for information coding in spectral differences in images. Some of these methods are implemented using digital image processing to alter pixel values within digital images in standard color formats. Of course, with greater control over the variables and materials used to render images, a much broader range of techniques may be implemented using spectral difference principles, yet extending them in rendering systems that offer access to more colors, and more control over rendering them into image output.
We also describe a variety of inventive methods and related hardware components for the complementary technology of determining spectral distributions, discerning spectral differences, and decoding information encoded in those spectral differences.
Our prior application Ser. Nos. 13/840,451 and 14/201,852, entitled Sensor-Synchronized Spectrally-Structured-Light Imaging, describe a variety of image capture technology and methods that can be used to determine a spectral distribution of colors at areas within an image. These application Ser. No. 13/840,451 (now published as U.S. Patent Application Publication No. 2013-0308045) and Ser. No. 14/201,852 (Now U.S. Pat. No. 9,593,982), are incorporated by reference in their entirety. The spectral distribution corresponds to measurements of the light energy within frequency bands. The particular capture and formation of the spectral representation can vary, and in particular, may be tuned for a particular application. The technologies in Ser. Nos. 13/840,451 and 14/201,852 for capturing spectral information are examples of techniques usable to discern information coding that uses the spectral differences to convey information within images. We describe its application to information coding and decoding in further detail below. We also describe other techniques for information coding and decoding in spectral differences.
One aspect of the invention is a method for encoding an information signal in spectral differences within an image signal, the method comprising:
generating an information signal;
mapping the information signal to locations within a host image signal; and
at the locations, inserting the information signal by computing values for color components at the locations so as to form a spectral difference signal that conveys the information signal within the host image, the spectral difference signal representing a distinguishable spectra between two or more colors that appear similar in the human visual system at the insertion locations within the host image signal.
In one embodiment, the distinguishable spectra comprise distinguishable spectra of a first color channel relative to a second color channel, which are metameric pairs. In particular, one example is where the first channel comprises a K channel, and the second channel comprises a combination of C, M and Y channels.
Another aspect of the invention is a method of decoding an information signal comprising:
obtaining spectra of an image signal at locations within the image signal;
discerning a spectral difference signal at the locations, the spectral difference signal representing a distinguishable spectra between two or more colors that appear similar in the human visual system at the locations within the host image signal; and
decoding an information signal from the spectral difference signal.
In one embodiment, the method of discerning the spectral differences comprises determining whether a spectra at a location is one of a particular set of distinguishable spectra. The particular set of distinguishable spectra may include a first spectra with a first shape, and a second spectra with a second shape. For example, the first shape is characterized by extent of variations over plural spectral bands (e.g., lumpiness), and wherein the second shape is characterized by absence of variations over plural spectral bands (relative flatness).
The first shape corresponds to a first combination of color channels, and the second shape corresponds to a second combination of color channels. For example, in one embodiment, the first combination comprises a K channel, and the second combination comprises a combination of C, M, and Y channels.
These are but a few inventive aspects disclosed in this document, and are not intended to be limiting to the various inventions described in the following description, which encompasses related work incorporated by reference.
The foregoing and other features and advantages of the present technology will be more readily apparent from the following Detailed Description, which proceeds with reference to the accompanying drawings.